Big Data Fusion Model for Heterogeneous Financial Market Data (FinDF)

The dawn of big data has seen the volume, variety, and velocity of data sources increase dramatically. Enormous amounts of structured, semi-structured and unstructured heterogeneous data can be garnered at a rapid rate, making analysis of such big data a herculean task. This has never been truer for data relating to financial stock markets, the biggest challenge being the 7 Vs of big data which relate to the collection, pre-processing, storage and real-time processing of such huge quantities of disparate data sources. Data fusion techniques have been adopted in a wide number of fields to cope with such vast amounts of heterogeneous data from multiple sources and fuse them together in order to produce a more comprehensive view of the data and its underlying relationships. Research into the fusing of heterogeneous financial data is scant within the literature, with existing work only taking into consideration the fusing of text-based financial documents. The lack of integration between financial stock market data, social media comments, financial discussion board posts and broker agencies means that the benefits of data fusion are not being realised to their full potential. This paper proposes a novel data fusion model, inspired by the data fusion model introduced by the Joint Directors of Laboratories, for the fusing of disparate data sources relating to financial stocks. Data with a diverse set of features from different data sources will supplement each other in order to obtain a Smart Data Layer, which will assist in scenarios such as irregularity detection and prediction of stock prices

Custom Hardware Versus Cloud Computing in Big Data

The computational and data handling challenges in big data are immense yet a market is steadily growing traditionally supported by technologies such as Hadoop for management and processing of huge and unstructured datasets. With this ever increasing deluge of data we now need the algorithms, tools and computing infrastructure to handle the extremely computationally intense data analytics, looking for patterns and information pertinent to creating a market edge for a range of applications. Cloud computing has provided opportunities for scalable high-performance solutions without the initial outlay of developing and creating the core infrastructure. One vendor in particular, Amazon Web Services, has been leading this field. However, other solutions exist to take on the computational load of big data analytics. This chapter provides an overview of the extent of applications in which big data analytics is used. Then an overview is given of some of the high-performance computing options that are available, ranging from multiple Central Processing Unit (CPU) setups, Graphical Processing Units (GPUs), Field Programmable Gate Arrays (FPGAs) and cloud solutions. The chapter concludes by looking at some of the state of the art solutions for deep learning platforms in which custom hardware such as FPGAs and Application Specific Integrated Circuits (ASICs) are used within a cloud platform for key computational bottlenecks